BRIDGE: Byzantine-Resilient Decentralized Gradient Descent

Machine learning has begun to play a central role in many applications. A multitude of these applications typically also involve datasets that are distributed across multiple computing devices/machines due either design constraints (e.g., multi-agent and Internet-of-Things systems) or computational/privacy reasons large-scale machine on smartphone data). Such often require the tasks be carried out decentralized fashion, which there is no server directly connected all nodes. In real-world settings, nodes prone undetected failures malfunctioning equipment, cyberattacks, etc., likely crash non-robust algorithms. The focus this paper robustification presence have undergone Byzantine failures. failure model allows faulty arbitrarily deviate from their intended behaviors, thereby ensuring designs most robust But study resilience within learning, contrast still its infancy. particular, existing Byzantine-resilient methods do not scale well models, they lack statistical convergence guarantees help characterize generalization errors. paper, scalable, framework termed Byzantine-resilient decentralized gradient descent (BRIDGE) introduced. Algorithmic for one variant BRIDGE provided both strongly convex problems class nonconvex problems. addition, experiments used establish scalable it delivers competitive results learning.